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INTRODUCTION 



O ver the past several years, a great debate has 

simmered in higher education about institutional 
accountability and performance. Efforts to alter federal 
policy in particular have been flashpoints for often 
heated, sometimes acrimonious discourse about how 
colleges and universities might best demonstrate their 
effectiveness. Accountability, access, and assessment 
are the buzz words of the day. And while much of the 
talk has had a decidedly “inside the Beltway” flavor, 
the discussions have in one way or another affected 
virtually every institution of higher learning across the 
country. 

In quiet counterpoint to the maelstrom over 
“policy,” a more measured approach has been at work 
on the campuses of a select group of colleges and 
universities. Over the past three years, some 33 liberal 
arts colleges and universities have been thoughtfully 
engaged in the challenging work of implementing a 
practicable way to measure student learning outcomes. 
These are the members of the Council of Independent 
Colleges/Collegiate Learning Assessment Consortium 



(hereafter, the CIC/CLA Consortium, or simply, the 
Consortium). These institutions have been hard at 
work using the Collegiate Learning Assessment (CLA) 
to gauge and improve student learning. 

The CIC/CLA Consortium has been an 
incubator of sorts, a place where hypotheses have 
been developed, tested, and challenged, out of 
which have come practices which in turn have been 
scrutinized and, if appropriate, adopted. Through their 
perseverance, the members of the Consortium have 
begun to demonstrate that the CLA is an effective, 
helpful, and meaningful tool to measure how the 
college experience helps students develop such 
higher order skills as thinking critically, reasoning 
analytically, solving problems, and writing effectively. 
In short, they demonstrate through practice that the 
CLA is an appropriate means to assess an institution s 
“value-added” contribution to learning over the course 
of a student s undergraduate education. 

This report is their story. 
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THE COLLEGIATE LEARNING ASSESSMENT (CLA) 



T he Collegiate Learning Assessment (CLA) 
was developed by the Council for Aid to 
Education (CAE) as a way of measuring institutional 
contributions to gains in student learning. The CLA is 
one of the first testing instruments to measure student 
learning directly by capturing the contribution of the 
institution to students’ cognitive development. The 
test provides a standardized measure of higher order 
skills and competencies that are often aligned with 
the general education goals of the undergraduate 
curriculum: critical thinking, analytic reasoning, 
problem solving, and written communication. 

Unlike most tests of student learning that use 
multiple choice, true ■'false, or short-'answer questions, 
the CLA uses written, student ■'constructed responses 
to open^'ended assignments. In contrast to subject ■' 
domain instruments that test students’ knowledge 
of particular disciplinary content, the CLA poses 
reahworld problems that students must address by 
analyzing materials, evaluating evidence, synthesizing 
information, drawing conclusions, and constructing 
their own arguments for or against a particular 
position. 

The CLA uses three key measures — developed by 
experts in psychometrics and thoroughly field^' tested — 
to assess student abilities: 

• Make an Argument* The ability to take and 
justify a position on an issue. 

• Critique an Argument* The ability to evaluate 
an argument for how welLreasoned it is. 

• Performance Task* The longest section of the 
test asks the student to complete a reahworld 
task, such as preparing a briefing report, using a 
set of provided materials. 

Thus, the test might state, “In our time, 
specialists of all kinds are highly overrated. We need 
more generalists — people who can provide broad 
perspectives,” and ask the test^'taker, in 45 minutes, to 
agree or disagree with the statement and explain the 



reasons for the position. Scoring rubrics, or criteria, 
provide a standardized basis for measuring test results. 

Using student SAT (or ACT) scores as a control, 
the CLA reports whether students, when measured 
as a group, perform at, above, or below expected 
levels. Thus, the CLA provides one of the first 
“value ■'added” measures that compares what students 
know when they start college with what they know 
when they finish (controlling for initial ability), and 
thus can demonstrate institutional contributions 
to student learning. This approach provides a 
reliable institutional value ■'added score that can be 
tracked over time and benchmarked against similar 
institutions. By making the institution, rather than 
an individual student, its primary unit of analysis, the 
CLA keeps the focus on how the institution as a whole 
contributes to student cognitive development. CLA 
results can be combined with other institutional data 
to determine factors that promote student learning and 
growth. 

The CLA yields two types of scores. An actual 
score shows how students performed relative to their 
ability — at, above, or below expected — for samples 
of first^'year and senior students. The value '•added 
score indicates how the aggregated first^'year to senior 
gains on these higher order skills compare with other 
institutions. 

The conceptual framework for the CLA has been 
thoroughly reported elsewhere. Richard Shavelson and 
Leta Huang (2003), for example, documented how 
the CLA has been informed by more than 100 years of 
efforts to assess student learning in higher education. 
Shavelson, Huang, Roger Benjamin, Stephen Klein, 
Marc Chun, and others have expanded and deepened 
our understanding of the CLA. In particular, Richard 
Hersh, the former president of Trinity College and 
Hobart and William Smith Colleges who now serves 
as a consultant to the CLA, has eloquently argued for 
and affirmed the value of the CLA in both the popular 
press and the higher education literature. Readers 
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Campus representatives from the 33 members of CICs Collegiate Learning Assessment (CLA) Consortium shared their 
progress in using the CLA to document student learning during a meeting on August 6 and 1, 2007, in Washington, DC. 



interested in the history, rationale for, and intellectual 
underpinnings of the CLA are invited to consult the 
resources referenced at the end of this report and on 
the CAE website, wwwxae.org. 

Finally, an important dimension of the CLA is that 
it is intended as one source of evidence in assessment 
of student learning, not as the sole measure. In the 
summer 2007 meeting of the CIC/CLA Consortium, 
for example, participating colleges and universities 
were urged to triangulate data from multiple sources 
to assess student outcomes. CAE program manager 
Alex Nemeth said that the CLA should be “part of a 
portfolio” of evidence of student learning. 

Similarly, in the spring 2007 issue of Peer Review, 
Richard Hersh wrote, “if useful learning assessment 
is the goal, multiple kinds of assessment are required, 
such as portfolios, comprehensive exams covering both 
general education and majors, thesis requirements 
(with and without oral examinations), and capstone 



courses....” (Hersh suggested, though, that any such 
combinations “are rarely utilized in a comprehensive, 
coherent, or cumulative way within any single 
institution.”) The notion of multiple approaches to 
assessment was also supported in the “Principles for the 
Uses of Assessment in Policy and Practice,” written by 
Lee Shulman, president of the Carnegie Foundation 
for the Advancement of Teaching. “Nearly any goal 
of using the results of assessment for serious practical 
and policy guidance,” Shulman wrote, “should 
intentionally employ an array of instruments.” 

Toward the triangulation of data, for example, 
several of the institutions in the CIC/CLA 
Consortium have experimented with pairing data from 
the CLA and the NSSE. Some institutions have also 
examined CLA results in the context of the Measure 
of Academic Proficiency and Progress (MAPP), 
another standardized outcomes test. 
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THE COUNCIL OF INDEPENDENT COLLEGES AND THE CLA 



C IC s work with the CLA began in 2002 when 
it was approached by the Council for Aid to 
Education to assist in identifying smaller private 
colleges to test the prototype of the Collegiate 
Learning Assessment. The following year, CIC 
recruited a group of 1 2 member colleges and 
universities to participate in the first year of public use 
of the CLA. In 2005, CIC 
expanded this initial group of 
institutions to include the 33 
colleges and universities that 
comprise the current CIC/ 

CLA Consortium, now in 
the final year of a three -year 
commitment to use the CLA. 

The goal of the CIC/CLA 
project is not to measure 
changes in individual 
students, but rather to learn 
more about programmatic 
features that correlate 
with “institutional effects” 
associated with larger 
than expected gains in 
students’ abilities to reason 
analytically, think critically, 
solve real-world problems, 
and write effectively. 

Consortium members 
commit to using the CLA 
on their own campus, 
administering the test to 
cross-sectional samples of 
first-year students in the 
fall and seniors in the spring, analyzing the results to 
determine areas of strength or weakness particular to 
their institutions and to develop appropriate strategies 
to improve student learning on their campuses. A 
team of faculty members and administrators from 
each member institution participates in an annual 



summer meeting of the Consortium, where they 
compare strategies for using the CLA and gain new 
understanding of its application. Upon return to their 
home campuses, those participants serve as advocates 
for the CLA and often become campus resource 
persons in assessing student learning. In addition. 
Consortium members share ideas and strategies 

throughout the year through 
web conferences, listservs, 
and email. 

“The CIC/CLA 
Consortium demonstrates 
many of the key features 
that characterize an effective 
approach to assessment 
work,” said Marc Chun, 
senior research scientist at 
CAE. “The participating 
campuses have all been firmly 
committed to using the 
CLA, and to do so for at least 
three years; this reinforces 
the notion that systematic 
assessment is not something 
that can be done casually. 

It is important for colleges 
to be actively involved 
in the work — to learn, to 
question, and to challenge 
and to be challenged. 1 
believe that participating 
in the Consortium has 
also raised the bar for the 
campuses. When campus 
representatives have the opportunity to interact with 
CLA psychometric, research, and program staff, the 
institutions have taken their work to the next level.” 

While the experiences of individual institutions in 
the Consortium and their respective paths to the CLA 
are unique, some patterns have emerged. At several 



The goal of the CIC/CLA 
project is not to measure 
changes in individual 
students, but rather to learn 
more about programmatic 
features that correlate 
with “institutional effects” 
associated with larger 
than expected gains in 
students’ abilities to reason 
analytically, think critically, 
solve real-world problems, 
and write effectively. 
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member institutions of the Consortium, presidential 
vision was the impetus that first brought the CLA to 
campus. At other institutions, the spark came from the 
vice president for academic affairs, often in concert 
with deans and, perhaps, professionals in institutional 
research. Across the Consortium, there are several 
cases where a faculty member emerges as a champion 
of the CLA, often after overcoming significant 
skepticism among colleagues about the test. 

At many institutions, the CLA has prompted 
vigorous and productive campus discussions about 
the practice of teaching, principles for learning and, 
at times, the institution s very mission. Also at many 
institutions, the CLA has proven to be a key driver of 
significant institutional reform. 

Roger Benjamin, president of CAE, praised the 
work of the CIC/CLA Consortium. “This was the 
first consortium of colleges formed to share best 
practice use of the CLA. This partnership has given 
my colleagues and me a unique opportunity to listen 
and learn from our colleagues at the participating 
institutions. In particular, the Consortium has helped 
us understand how to harness the case study approach 
focus of the performance tasks which can be used 
by faculty members to directly help them improve 
teaching and learning,” Benjamin said. 

It is worth noting that the CLA does not exist in a 
political vacuum. The assessment of student learning 
has come to the fore in the midst of sometimes heated 
debates about the accountability of higher education. 
Some resistance to the CLA is linked to controversial 
political machinations around accountability on 
the part of the U.S. Department of Education under 
the Bush administration. In the 2006 report, A Test 
of Leadership: Charting the Future of US. Higher 
Education^ Secretary of Education Margaret Spellings’ 
Commission on the Euture of Higher Education stated, 
“we believe that improved accountability is vital 
to ensuring the success of all the other reforms we 
propose. Student achievement, which is inextricably 



connected to institutional success, must be measured 
by institutions on a Value-added’ basis that takes into 
account students’ academic baseline when assessing 
their results.” The report called for this information to 
be shared publicly. 

ClC’s efforts to establish the CIC/CLA Consortium 
predate the Spellings Commission report by several 
years. Nonetheless, the CLA was drawn into this 
national debate by virtue of it having been discussed 
extensively in the report. Specifically the report said: 

Among the most comprehensive national 
efforts to measure how much students actually 
learn at different campuses, the Collegiate 
Learning Assessment (CLA) promotes a 
culture of evidence-based assessment in higher 
education.... The CLA allows for comparability 
to national norms and measurement of value 
added between the freshman and senior years. 
Additionally, because the CL As unit of analysis 
is the institution and not the student, results 
are aggregated and allow for inter- institutional 
comparisons that show how each institution 
contributes to learning, [p. 23] 

The report went on to recommend that, “higher 
education institutions should measure student learning 
using quality assessment data from instruments such 
as, for example, the Collegiate Learning Assessment, 
which measures the growth of student learning taking 
place in colleges....” Thus, even though the CLA 
is itself a politically neutral instrument, it has been 
politicized through the Commission’s report. 

As members of the CIC/CLA Consortium have 
learned, the CLA is not a flawless instrument. 
Institutions that insist that any measure of student 
learning be perfect before using it will find something 
not to like about the CLA, as they would likely find 
with any instrument. The CIC campuses that have 
successfully pioneered the use of the CLA have 
recognized that it is not a panacea, but it is a strong 
tool that can help improve teaching and learning. 
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THE IMPORTANCE OF CAMPUS LEADERSHIP 



The impetus for a campus to become involved in the CIC/CLA Consortium has come in some instances from 
the institutions leaders. Allegheny College and the University of Charleston provide good cases in point. 



Allegheny College 

Located 90 minutes north 
of Pittsburgh in Meadville, 
Pennsylvania, Allegheny 
College is the oldest 
college in the nation. Its 
student body numbers 
around 2,100. Among 
many distinguishing 
characteristics, Allegheny 
requires that every 
student declare a 
disciplinary major as well 
as a minor outside the division of the major — thus 
offering more than 900 disciplinary combinations for 
students to expand their horizons. Each Allegheny 
student completes a senior project in his or her major 
field, a significant piece of original research that 
often becomes the core of the student s portfolio of 
accomplishment. 

Richard Cook, Allegheny’s president, said he 
learned about the CLA from Richard Hersh. “That’s 
where my original interest was sparked,” Cook says, 
“and when the CLA went into its early phases of going 
more widely, 1 asked that we participate in it.” Cook 
realized how the CLA could be a promising tool for 
assessing general learning outcomes for Allegheny’s 
students. Once he brought the idea to campus, though, 
he says that “others took ownership of it. It wasn’t too 
hard. We have people here, thank goodness, who really 
care about teaching and learning.” 

One of the people who took ownership of the 
idea is Marian Sherwood, Allegheny’s director of 
institutional research. She joined the Allegheny 
community 12 years ago, at about the same time that 



Cook was appointed president. She recalls a talk 
Cook gave early in his tenure in which he said that 
“students’ self-reported data” is one way to measure 
educational effectiveness but that Allegheny needed 
something better, “something that isn’t just what the 
students think about what they’ve learned.” So from 
Cook’s early years as president, she says, Allegheny was 
“looking for something that would be a direct measure” 
of student learning outcomes. 

Allegheny was quick to get involved with NSSE, 
Sherwood recalls, but still continued to struggle with 
the question, “how do you get a direct measure of 
what students do?” And that, she says, “is a harder 
nut than surveying the students with questions about 
either their experiences or their perception of their 
experiences.” In that light, the CLA looked very 
attractive indeed (when it became available). 

Allegheny joined the CIC/CLA Consortium on 
the heels of its accreditation review by the Middle 
States Commission on Higher Education. At roughly 
the same time, the campus community was involved 
in a national discussion about the efficacy of rankings 
provided by the magazine U.S. News and World Report. 
As a whole, the campus found that the magazine’s 
assessment of so-called “input” measures of student 
learning were insufficient to measure the quality of 
an Allegheny education. The college wanted to find 
a more substantive measure of how an Allegheny 
education contributes to the development of a 
student’s skills in critical and analytical thinking and 
in communication. 

Linda DeMeritt, dean of the college at Allegheny, 
says that Allegheny has seen “increased attention to 
assessment” overall and believes that the CLA has 
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“I hope that this is something that other colleges will pursue and 
that we’ll all continue to learn about because I think it’s vitally 
important, both for demonstrating value added and for improving 
what we do.” 



-Richard Cook, President, Aiiegheny Coiiege 



been “an important component of that increased 
attention.” For its Middle States review, DeMeritt 
says Allegheny “outlined an extensive process for 
evaluating departments through self studies, learning 
outcomes of students, and the effectiveness of 
individual courses for the major and for the minor.” 
But the college also wanted to assess its senior project 
and saw the CLA as a potential way to accomplish 
that goal. Gradually, the scope of Allegheny’s focus 
expanded to a point where it began to view the CLA 
as a tool for assessing the whole of its curriculum. The 
CLA became “one of the pieces in our thinking of an 
overall assessment plan for the institution,” DeMeritt 
says, in part because “it focuses on what we try to do 
in our general education requirements, in our majors, 
in our minors, and in the senior project — namely 
to educate our students to think critically, analyze 
problems, find solutions, and communicate in written 
form persuasively and logically.” 

Allegheny reports that its use of the CLA has not 
always been smooth. Its initial results showed that 
student skill levels were not at the levels the college 
expected, nor were they in alignment with more 
positive results from other measures. That prompted 
the institution to look more deeply, for one thing, at 
logistics in administering the test. Allegheny found 
that its results were skewed to some extent by the 
fact that incoming first-'year classes were progressively 
stronger academically through the years in which 
it first administered the test. “Our academic profile 
has gone up in the last three or four years,” Richard 
Cook says, “and so we haven’t had an apples to apples 
comparison. We think that puts uncertainty into the 
system.” At the same time, he observes, “it’s too early 



to have a cycle of testing the very same cohort of 
students as incoming students and as seniors.” 

Allegheny doesn’t intend to be deterred by bumps 
in the road to adopting the CLA. As the college wrote 
in a report to CIC, “we can deal with the logistics of 
administration, and we want to gather additional data 
to learn what we can about the educational experience 
at Allegheny. Both of these tasks can be more easily 
and fruitfully accomplished in a Consortium of like' 
minded institutions of higher education willing to 
share their ideas and experiences.” 

It’s too early to talk about the potential impact of 
the CLA on the content of Allegheny’s curriculum, 
but Linda DeMeritt says that as Allegheny reviews 
syllabi and teaching effectiveness, it is asking whether 
it can “incorporate any of these types of performance' 
based tests into our own pedagogy.” In that sense, 
she says, the CLA is having an impact by prompting 
Allegheny’s faculty “to think more in terms of learning 
outcomes than teaching objectives.” 

“We see the CLA as still a work in progress,” Cook 
says, “promising, but not yet definitive.” But even 
if it’s “not a perfect measure,” he says, “we think it’s 
worth pursuing, and it’s a direct measure of student 
performance. That part of it appeals to us.” Cook 
believes that “as we gather more data over the years, 
those data will become more reliable and more of use 
to us.” 

“1 hope that this is something that other colleges 
will pursue and that we’ll all continue to learn about 
because 1 think it’s vitally important,” Cook says, 

“both for demonstrating value added and for improving 
what we do.” 
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University of Charleston 

Another college leader 
who has been an advocate 
for the CL A is Edwin H. 
Welch, president of the 
University of Charleston. 
Located on the banks 
of the Kanawha River 
in West Virginia s state 
capital, the university 
enrolls some 1,200 
students. Almost a decade 
ago, Welch says, the 
university began asking hard questions of itself. “We 
looked at the future of the institution,” he says, and 
saw that “we weren’t well positioned to be successful.” 
Moreover, the university felt a strong need to identify 
an educational niche that would distinguish it from 
other institutions. 

From the vantage point of a seat on the board 
of a major regional health facility, Welch had seen 
how outcomes-based quality improvement programs 
were a regular and invaluable mechanism for 
institutionalizing enhanced practice and performance 
in a panoply of departments and applications. Indeed, 
Welch noted, assessing, documenting, and improving 
quality was a given requirement in the healthcare 
environment. He immediately saw that similar 
practices could be adapted to higher education. 

Following many months of institutional self- 
assessment at the University of Charleston, a campus 
“visioning” committee concluded that UC would 
claim a niche in higher education focused on outcomes 
education, both as a way to improve quality and 
to differentiate itself from other institutions. The 
university rewrote its curriculum with a focus on 
core competencies and learning outcomes in six key 
areas: citizenship, communication, creativity, critical 
thinking, ethical practice, and science. 

The university jumped at the chance to take part in 
the CLA, Welch says. The test fits well at UC, he says, 
“because we are trying to assess the same thing — what 
do students know when they come in, what are we 



helping them to learn, how well can they demonstrate 
critical thinking, and so on.” The test also was proof 
of the effectiveness of the direction in which the 
institution was headed: In 2005-2006, the university 
had the highest value-added score among the more 
than 100 colleges and universities that administered 
the CLA. 

Welch credits the CLA as one of the core factors 
“that has changed our understanding of learning.” 
Specifically, he says, the CLA has helped the 
institution affirm that it is on the right track in 
wanting to “find out whether people really learn 
something in college.” The test has also helped the 
university identify aspects of learning, such as critical 
thinking, in which the university wasn’t as successful 
as it should be and needed more focus. Taken 
seriously on campus and widely discussed, the CLA 
has been an effective tool to prompt “reevaluation 
and reassessment” of what’s working and what’s not 
working when it comes to learning, Welch says. 

“Using the best available tests for measuring 
student learning and combining those with self-created 
instruments,” Welch says, “the university is providing 
evidence to students and parents about the process and 
the results of a University of Charleston education. 
Improving student learning is not just good for the 
institution; it is precisely what students and parents 
deserve.” 

Welch also views UC’s drive to develop a culture 
of assessment in the context of discussions at the 
level of federal policy. “1 guess I’m less critical than 
most of my colleagues about the push for validation 
of learning — 1 think the federal government has a 
right to know how well we’re using their money.” 

He’s quick to add, though, that “1 don’t want to give 
the government a blank check in any way, shape, or 
form. We’re a private institution and 1 respect that 
independence.” At the same time, Welch says, “if we 
find good instruments, then it’s okay to teach to the 
instruments, because they’re measuring what you want 
students to have. And 1 don’t find that threatening. 1 
find it challenging.” 
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THE VARIETY OF CLA EXPERIENCES 



Experiences of CIC/CLA Consortium members have shown the Collegiate Learning Assessment to he an 
effective instrument for a wide variety of institutions . 



Lynchburg College 

The CLA has had an 
impact on the campus 
of Lynchburg College, 
an institution of 2,400 
students in in the 
foothills of the Blue Ridge 
Mountains in Lynchburg, 
Virginia. Lynchburg is 
one of just 40 institutions 
featured in Loren Pope’s 
book. Colleges That Change 
Lives, “The CLA has 
challenged us to look at what we do to determine the 
value that we add to the critical thinking ability of 
our students,” the institution wrote in a report to CIC. 
“Prior to participating in the CLA we said what most 
colleges and universities say — that ‘an LC education 
is worth the time, effort, and money invested.’ Now 
we have data from an outside agency to confirm our 
assertion about the value of an LC education.” 

Deborah Driscoll, Lynchburg’s associate vice 
president for institutional effectiveness, planning, and 
assessment, says that the CLA was first introduced 
on campus in the context of an institutional strategic 
planning process. When Driscoll learned about the 
CLA at a professional conference, she saw immediately 
that the test could be a good way to verify whether the 
effect that Lynchburg said it had on students was really 
accomplished. She worried, however, that test results 
might not in fact reflect well on the college. 

Lynchburg was pleased with its initial CLA results. 
“It didn’t surprise us that our freshmen tested a little 
bit below expectations,” Driscoll says, because first- 
year students have “just reoriented their world — 



they’re trying to figure out what the rules are, what’s 
going on.” The results from seniors, however, told a 
very strong story of success. 

“Our seniors did well,” Driscoll recounts. “In the 
second year of our results, they put us in the top 
10 percent of value added, the differences between 
freshmen coming in and seniors going out.” Lynchburg 
felt especially good about these results in light of the 
college’s challenging curriculum. 

Lynchburg’s president, Kenneth R. Garren, wrote 
about the institution’s CLA findings in an op-ed 
published in September 2007 in a local newspaper. 
Speaking about the CLA in the context of national 
ratings of colleges and universities, Garren wrote, 
“many would argue. . .that although the hard data 
collected on each institution would appear to be a 
logical indicator of an institution’s excellence, it does 
not necessarily mean that particular students are being 
well-served by the institution.” Garren suggested that 
the CLA provided a valuable method for digging into 
relevant questions that could yield deeper insights: 
“How well will a college perform in providing the most 
beneficial educational experience for the prospective 
entering student?” and “Is there a way to actually 
measure with any degree of certainty the ability of 
a college to offer a value-added component to the 
educational experience of its students?” Or, putting 
it another way, “what happens after students get to 
college?” 

“One of our challenges,” Driscoll says, “is to get the 
information to our faculty in a way that helps them 
think about what they’re doing.” CLA results, she says, 
will help Lynchburg “stimulate those conversations 
and share information.” 
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As a member of the CIC/ 
CL A Consortium, Ursinus 
College links its interest 
in the CLA to growing 
faculty involvement 
in assessment. Also 
included in Colleges That 
Change Lives, where it 
is dubbed a “star of the 
first magnitude,” Ursinus 
is located some 30 miles 
from Philadelphia and 
enrolls 1,600 students. The colleges Committee on 
Outcomes Assessment works with faculty members 
to draw insights from their CLA results that can 
inform classroom learning. Of particular interest 
is determining the impact on student intellectual 
development of the first-'year liberal studies seminar. 
The Common Intellectual Experience (CIE), and how 
it can be used to gauge the effectiveness of a pilot “CIE 
for Seniors” that is in development. The college is 
also interested in developing opportunities for faculty 
members to create CLA-type testing in their own 
classes. 

Westminster College 

Westminster College, 
in Eulton, Missouri, also 
views the CLA as a means 
to improve teaching and 
learning at the course and 
program level. A highly 
selective institution 
of just over 1,000 
students, Westminster 
is the site of Winston 
Churchill s famous 1946 
“Iron Curtain” speech. 
Westminster pursues what it calls an aggressive agenda 
for assessment, a foundation of which is “knowing and 
being able to articulate the difference we make in the 
lives of students.” 



At the beginning of the 2007-2008 academic 
year, Westminster was beginning its fourth year 
of involvement with the CLA. Already fairly 
sophisticated in its use of assessment instruments, the 
college is now looking at ways to draw on the strengths 
of the CLA to enhance work in the assessment of 
writing, a skill for which faculty members have learned 
to trust assessment analyses and to use that feedback to 
improve teaching. 

More sophisticated analysis of CLA results is also 
seen as a way for Westminster to understand better 
the relationships between students’ engagement in the 
learning process and their ability to think, reason, and 
solve problems. In the fall 2007 semester, the college 
for the first time matched students’ responses to NSSE 
with measures of performance on CLA tasks. Enough 
significant relationships were identified to encourage 
the institution to explore this line of inquiry further. 
Westminster is also starting to explore ways by which 
campus discussions of the CLA can serve as a vehicle 
for enhancing the communication of assessment results 
to students and other campus constituencies. 

Texas Lutheran University 

Texas Lutheran University 
has found the CLA to 
be an excellent tool 
for helping faculty 
members to improve their 
understanding of the value 
of outcomes assessment. 
Texas Lutheran is located 
in Seguin, Texas, near 
San Antonio and Austin, 
and has a student body 
of 1,400. Apart from the 
fact that TLU ranks well in national measures based 
on “inputs,” provost and executive vice president 
John Masterson says that “one of the great appeals 
of the CLA is that it actually measures some student 
outcomes. It’s the right kind of assessment.” But the 
benefits of CLA also have deeper meaning for TLU. 



Ursinus College 
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Another powerful aspect of the CLA, says 
Masterson, is that it is helping diverse campus 
constituencies, including faculty and staff members, 
come to a deeper appreciation that “assessment 
is not just something you do for the accrediting 
bodies/’ Instead, through campus discussions of 
the CLA, TLU faculty members are beginning to 
recognize how assessment can help them enrich 
their teaching and improve student learning. The 
result is that through such strategies as strengthened 
program review, better institutional effectiveness 
procedures, participation in the NSSE survey, and 
hard work by internal committees, TLU is creating 
what Masterson calls “a real culture of assessment and 
continuous improvement” that is more clearly focused 
on improving the quality and structure of student 
educational experiences. 

TLU is also experimenting with another possible 
use of the CLA. With an interest in improving its 
ability to retain students, especially first-'year students, 
TLU has been scrutinizing its CLA results to see what 
clues they offer about retention. Nick Lockard, dean 
of the university’s college of professional studies, says 
TLU has been looking, for example, at whether there 
is a relationship between a student’s low score on the 
CLA and the student’s decision not to return after 
the first year. The university will analyze the data 
to determine if there are patterns in overall results 
and such factors as student majors and grade point 
averages. 

Alaska Pacific University 

Thousands of miles from 
Texas, Alaska Pacific 
University is moving in 
similar directions. APU 
is one of the 49^^ state’s 
two independent four- 
year colleges. Billing 
itself as “above the 
ordinary in more than 
latitude,” APU says it is 
“a place for activists and 



idealists.” Many of APU’s first-year undergraduate 
students come from outside the state, attracted by 
Alaska itself and by the institution’s strong offerings 
in environmental science and marine biology, one 
of few such undergraduate programs in the country. 

The institution also attracts a fair number of Alaskans 
who leave the state for their first years of college, 
then transfer to APU as sophomores and juniors. The 
university’s undergraduate program makes up not 
quite half of the total student population; an evening 
degree-completion program and master’s-level study 
in counseling psychology and business fill out the 
institution’s student body — which totals a head count 
of only about 700. 

Over the last four years, APU has moved to 
embrace assessment based on learning objectives 
across all of its academic programs. The faculty 
adopted a unified assessment plan in 2005 as part 
of APU’s self-study for an accreditation visit by the 
Northwest Commission on College and Universities 
(NWCCU). The university’s accreditation was 
reaffirmed in 2006, but NWCCU plans a follow-up 
visit in 2009 to evaluate APU’s progress in assessment. 
APU was invited to join the CIC/CLA Consortium 
while the university was re-inventing its assessment 
effort. University officials say the CLA has been a 
key element in focusing its assessment efforts and 
promoting related faculty development. 

Marilyn Barry, APU’s academic dean, says the 
university has been “trying to be intentional and 
reflective about what we’re doing and why we’re doing 
it.” Barry has introduced concepts from the CLA to 
help nurture this process among the faculty. The CLA 
complements, for example, a recent push by the faculty 
to integrate rubrics for student learning objectives 
into syllabi. Using rubrics as tools of analysis to help 
students synthesize learning, Barry suggests, has been 
instrumental in helping faculty members enhance their 
pedagogical effectiveness. APU has found that CLA- 
based exercises are effective ways of both focusing 
faculty development programming and nurturing 
faculty leadership. 
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APU faced some distinctive challenges in using the 
CLA. Many of its seniors, for example, did not have 
either SAT or ACT scores, a standard benchmark 
for the CLA. CLA administrators worked to find an 
alternative test. The Scholastic Level Exam (SLE), 
a short-form measure of cognitive ability produced 
by Wonderlic, proved acceptable as a substitute 
benchmark. With its small student body, the 
institution has struggled to survey enough students 
to generate viable CLA scores. The CLA works for 
APU s undergraduate population but does not apply 
as well to its other students. And APU s curriculum 
has posed big challenges as faculty members developed 
rubrics to measure student learning. Alaska Pacific 
is focused on “teaching and trying to do well by 
our students and our profession,” Barry says. As the 
university builds its strategy for improving student 
learning on campus and works to weave assessment 
with goals for learning outcomes, she anticipates that 
the CLA will continue to be an integral benchmarking 
tool. 

Stonehill College 

At Stonehill College, the 
CLA has been an impetus 
for proposed changes in 
the curriculum specifically 
designed to advance 
student learning. Stonehill 
is a Catholic institution 
that enrolls nearly 2,400 
students on its campus in 
Easton, Massachusetts, 
near Boston. Initial results 
from both the CLA and 
NSSE — in combination with increasingly selective 
admissions practices — led the college to question 
whether it was adequately challenging its students. 
Data from test results proved important reference 
points in a campus discussion that eventually led to 
a formal proposal to modify Stonehills course-credit 
model in part to create opportunities to increase 
academic challenge and rigor. 



Data from various assessment instruments 
administered at Stonehill, including the CLA, also led 
to proposed changes in the college’s general education 
program, including the development of first-year 
seminars that would focus on disciplinary knowledge 
and the development of critical thinking and writing 
skills. In addition, Stonehill is in the process of a 
comprehensive review of the senior capstone course in 
each major to ensure that learning outcomes originally 
established for those courses are being achieved. 

Seton Hill University 

One more example of 
the CLA in practice 
comes from Seton Hill 
University, a Catholic 
institution 35 miles east 
of Pittsburgh that has 
about 1,800 students. The 
university has an effective 
program review cycle 
for all academic majors 
and graduate programs, 
undertakes assessment 
routinely as part of accreditation processes, and 
annually conducts a review of the learning objectives 
in its liberal arts core curriculum. 

In part prompted by results from early experiences 
with NSSE and later with the CLA, the campus 
found itself in conversation about what value it 
was really adding for students. The NSSE results 
had strongly suggested that students at Seton Hill 
did not think they were sufficiently challenged. Eor 
example, students themselves wrote that they didn’t 
think they were reading enough or writing enough 
papers. The university’s first CLA results were also 
disappointing, prompting the university to engage 
in many conversations about how it could improve 
student skills in critical thinking and other areas that 
CLA measures — and, importantly, how it might make 
relevant changes in pedagogy. Among other responses, 
the university’s faculty launched a year-long series 
of bi-weekly professional development sessions for 
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“The ongoing relationship with the Consortium schools has created 
a safe environment for individual campuses to share their successes 
and challenges. It has also served to be a testing ground and 
incubator of new ideas and new initiatives.” 

—Marc Chun, Senior Research Scientist, Council for Aid to Education 



faculty members under the theme “Teaching Critical 
Thinking Skills.” Faculty members also voted to 
approve a new writing- intensive component in which 
every major is expected to have at least one course 
designated as writing intensive. Faculty members also 
took part in a semester- long seminar on the teaching 
of writing in the disciplines. 

Since these adjustments were made, results on both 
the CL A and NSSE have been more promising. That 
in turn has encouraged the institution to do even 
more. In essence, Mary Ann Gawelek, vice president 
for academic affairs and dean of the faculty, says the 
CLA opened a door that allowed the campus to look 
at a lot more information about its performance. 
Additionally, “the CLA data have forced us to ask 
ourselves questions.” For example, Gawelek says, 
faculty members and administrators are asking, “How 
do we really work and push ourselves to use more 
creative, more applied assessment techniques that 
would move this agenda?” She adds that having the 
CLA data in hand is a powerful motivator. 

“We report both NSSE and CLA data at our 
opening fall workshop, so everybody hears the 
information, not just the faculty — including, by the 
way, our student leaders,” Gawelek says. The university 
has also shared test results with the educational policy 
committee of its board. Trustees, Gawelek says, have 
asked penetrating questions about how the university 
uses test results. 

The CLA experience has also had a ripple effect 
in campus conversations beyond the faculty — 
essentially forcing discussions of learning out to “our 
nonclassroom environments,” Gawelek says. Seton 
Hill has become “adamant about looking at learning 



outside of the classroom” with the same kind of 
scrutiny it brings to in-class learning, she says. “What 
do our clubs do? How do we teach maturity in the way 
students handle their financial aid or work study?” 
Seton Hills student life staff, for example, has created 
a series of learning objectives, related to university 
objectives, for resident assistants. 

In short, Seton Hill believes it has made significant 
progress in assessment but that more work needs to 
be done. The university is interested, for example, 
in designing a comprehensive faculty development 
program that prepares faculty members to create CLA- 
type assessment measures that apply to their disciplines 
and that demand use of critical thinking and analytic 
reasoning. The CLA would be one of the university’s 
key models for the design of assessment measures. 

The Consortial Approach 

In addition to important lessons learned on 
individual campuses, the Consortium experience 
of gathering for annual summer meetings to share 
successes as well as problems has yielded other benefits. 
“The consortial approach has been both a campus as 
well as a collective effort; by meeting annually and 
by maintaining means of communication (such as 
through a listserv), the level of understanding and 
commitment has been notably enhanced,” said Marc 
Chun, senior research scientist at CAE. “The ongoing 
relationship with the Consortium schools has created 
a safe environment for individual campuses to share 
their successes and challenges. It has also served to be 
a testing ground and incubator of new ideas and new 
initiatives.” 
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EXAMPLE-BARTON COLLEGE 

“APPLYING THE CLA TO HELP CHANGE THE CAMPUS CULTURE” 



L ocated in Wilson, North Carolina, Barton College 
is an academic community of approximately 
1,300 students and 200 faculty and staff. Barton is 
recognized particularly for its programs in education, 
deaf education, nursing, and social work. Barton is 
in its third year of using the CLA. The institution’s 
experiences with the CLA have informed its strategic 
planning process and discussions with not just trustees, 
faculty members, and staff members, but also with 
prospective students and their parents. 

Terrence L. Grimes, Barton’s vice president for 
academic affairs, says that Barton’s involvement in the 
CLA came at an opportune time for the institution, 
coinciding with institutional self-assessment in 
preparation for review by the Southern Association of 
Colleges and Schools (SACS). Grimes saw the CLA 
as a tool that could help Barton address concerns he 
had about its general education program. Essentially, 
the college’s work with the CLA proved to be a way 
to draw administrators, faculty and staff members, 
and even trustees into a frank, sometimes difficult 
but ultimately constructive conversation about how 
well Barton was actually educating its students. The 



ultimate effect, as it turns out, is that Barton’s use of 
the CLA is helping to shift the institution’s culture. 

Before Barton got involved in the CLA, the 
college’s approach to assessment was typical of most 
colleges. Grimes says, in that it was “unscientific.” 
Grimes recalls writing portfolios that were never 
collected, let alone analyzed, and courses that 
ostensibly emphasized writing but “really didn’t.” In 
short, he says, there was very little concrete evidence 
of actual performance. 

Barton’s initial experience with the CLA proved 
highly instructive. Test results showed evidence that a 
Barton College education actually added value but also 
provided the impetus for the college to do even better. 
Powerful lessons, for example, came from samples of 
first-year and senior writing that the CLA provided. 
The college used extra-credit-point incentives to 
encourage student participation in the CLA. To make 
sure that results were not unduly influenced by the 
incentives, some students were assigned to take what 
Grimes dubbed as “placebo” tests, locally administered 
and not used to compute the institution’s CLA results. 
The alternative test consisted of two prompts that had 
been retired by CLA, exercises 
respectively in how to make and 
how to break an argument. Grimes 
took it upon himself to score the 
responses from the alternative 
samples, using rubrics patterned 
after the CLA. 

The results were eye-opening. 
“While we can honestly say that 
the CLA provides good evidence 
that our students improve in their 
critical thinking and writing skills 
during the time that they attend 
Barton College,” Grimes observed, 
student performance on the writing 
tests “left something to be desired.” 
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“The college’s work with the CLA proved to be a way to draw 
administrators, faculty and staff members, and even trustees into a 
frank, sometimes difficult but ultimately constructive conversation 
about how well Barton was actually educating its students.” 

-Terrence Grimes, Vice President for Academic Affairs, Barton Coilege 



Grimes began sharing what he had learned with 
campus audiences. The timing was fortunate and 
the results were noteworthy. Barton s strategic plan 
calls for the college to have a new general education 
curriculum with a focus on engaged learning in place 
by the fall of 2008. Also, Barton s SACS review 
takes place during the spring and fall of 2008 — and 
general education assessment is a major emphasis in 
the reaccreditation process. Toward those endpoints, 
the Barton faculty had been involved for two years 
in identifying and discussing ways to assess the 
learning outcomes for general education, outcomes 
that included critical thinking and communication. 
Grimes shared the hard evidence of students’ written 
responses from the placebo test with the college’s 
general education task force. The task force proposed 
a new curriculum that included a significant emphasis 
on writing and critical thinking. Precisely because 
“it’s based on learning outcomes,” Grimes calls the 
proposed new curriculum a significant change. 

The CLA results have also helped advance Barton’s 
curricular improvements by informing the creation 
of its quality enhancement plan (QEP), a SACS 
requirement. Barton’s QEP committee focused on the 
need for the college to bolster student writing skills 
after they saw the results of writing and thinking 
abilities that the CLA revealed. Barton’s CLA 
experience also dovetailed. Grimes says, with work it 
had started to enhance its students’ focus on engaged 



learning — including “some very significant changes in 
pedagogy,” designed to deepen student involvement in 
their own learning and “focus more on outcomes and 
collecting evidence” — the result of work it had done 
with the National Survey of Student Engagement. 

As part of an effort to help Barton’s trustees 
better understand the college’s curricular planning 
and preparation for its SACS visit. Barton President 
Norval C. Kneten and Grimes also shared the seniors’ 
writing samples with Barton’s board of trustees. The 
trustees were given the relevant rubrics and asked to 
assess the student papers. Like virtually everyone else 
on campus who had seen the papers, the trustees were 
disappointed by what they read. Through its academic 
affairs committee. Barton’s board adopted a strong 
statement urging faculty members to develop strategies 
for improving student skills in written communication 
and critical thinking. 

“We’re using the CLA to change the campus 
culture,” Grimes says. “We can say that our students 
perform at the level expected — that’s perfectly 
honest — and there is value added in our education. 

But are our students really being prepared to go out 
into the workforce or graduate institutions with 
writing and critical thinking skills?” Grimes is relying 
on Barton’s use of the CLA to help the college refocus 
its curriculum in a way that ensures that it can respond 
affirmatively and enthusiastically to that important 
question. 
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EXAMPLE-CABRINI COLLEGE 

“THE QUEST FOR ‘INTENTIONAL ENHANCEMENTS’” 



C abrini College, a Roman Catholic, liberal 
arts college in the suburbs of Philadelphia, 
Pennsylvania, with an enrollment of 2,300 
undergraduate and graduate students, has a deep and 
abiding interest in the assessment of student learning. 
Like many other institutions, however, it had only 
dabbled in assessment until recently, using ad hoc 
strategies and far-from-perfect assessment instruments. 

Charlie McCormick, Cabrini s dean for academic 
affairs, reports that the college has used the National 
Survey of Student Engagement instrument and data 
from UCLA’s Cooperative Institutional Research 
Program (CIRP) first-'year survey. In addition, he says, 
Cabrini has “developed some of our own internal 
materials for student expectations surveys and our 
departments have produced departmental assessment 
plans.” However, the home-grown instruments varied 
in quality, he says, and there wasn’t much uniformity 
in the way they were used. Overall, the college’s 
assessment efforts lacked coherence. 

Cabrini wanted to systematize assessment of 
student learning, improve its practice, and make it 
a regular part of the institution’s ethos. The college 
wanted to move beyond ad hoc assessment and design 
new approaches that explicitly and seamlessly feed 
back into a quality enhancement loop. It wanted to 
find meaningful ways to use assessment data to make 
what it called “intentional enhancements” in student 
learning. Cabrini wanted, in other words, to find a way 
to inculcate a workable culture of evidence that would 
lead to regular, meaningful assessment of student 
learning. Against this backdrop, Cabrini became a 
member of the 2005-2008 CIC/CLA Consortium. 

Getting involved in the CIC/CLA Consortium 
proved pivotal for Cabrini not only because it 
introduced the college to an effective assessment 
instrument, but because it provided a focus for the 
college’s assessment efforts as a whole. Prior to Cabrini 
joining the Consortium, McCormick says, “nothing 
had emerged to help center our assessment efforts. 



There were a lot of things going on in a lot of places, 
and we didn’t have an anchor on which to tie these 
various other assessments.” The Collegiate Learning 
Assessment filled that gap, he says, and, “really 
emerged as that sort of anchor.” 

Cabr ini’s initial CL A test scores, for example, 
showed promising results. “Even though our final 
absolute scores weren’t as high as we wanted them 
to be,” McCormick says, “we found out we were a 
value-added institution.” In other words, students 
were demonstrating important gains in higher order 
skills as a result of their undergraduate education 
at Cabrini. With tangible results from the CL A in 




THE COUNCIL OF INDEPENDENT COLLEGES *19 




hand, McCormick says, Cabrini could point to the 
assessment and say, “we’re really doing something 
here.” 

Evidence from the CLA supported Cabrini’s 
mission to foster student development. “Our president 
likes to say ‘we get good students and we make them 
great students,’ ” McCormick says, “and we can see 
in our value-added scores that wasn’t just rhetoric.” 
When shared with multiple college constituencies, 
from the faculty to enrollment staff to the board, the 
CLA data had an important, positive ripple effect 
across campus, helping the college community see that 
Cabrini was competitive with other institutions in 
ways that hadn’t been clear before. 

While Cabrini’s leaders were quick to note that 
all was not rosy — the college readily recognized 
the areas in which it must improve — McCormick 
reports that one key effect of the CLA results is 
that they gave the college “a new way to talk about 
ourselves, a new way to talk about what it was that 
our institution was doing.” That in turn energized a 
new kind of conversation on campus. The CLA data 
provided a way for the college to share a more nuanced 
story about Cabrini’s educational experience with 
both prospective students and those who had been 
associated with the college for a long time. 

McCormick and colleagues also began to explore 
ways in which it could further expand its assessment 
of student learning by comparing results from the 
CLA with those from NSSE. “We began to have 
that conversation with department chairs,” he says, 
“asking them to go back and share the results with 
their departments.” Importantly, he notes, dialogues 
on campus started addressing seminal questions 
about what really mattered to Cabrini. Cabrini’s 
administrators asked faculty members to “think 
about how they could make programmatic changes,” 
McCormick says, that could lead to scores on the 
assessments that would “reflect the areas where we 
wanted them to reflect positively,” in alignment with 
the college’s mission. 

Laculty members were a key focus of these 
conversations, of course, but Cabrini also made 



a point of discussing the assessments with the 
president’s cabinet, the enrollment management 
committee, staff members in financial aid, and other 
campus constituencies. Talking about the test results, 
McCormick says, opened up a way to engage the whole 
campus in a conversation about where the college 
wanted to go. Moreover, McCormick says, it created 
a means of engaging campus staff other than faculty 
members in meaningful conversations about what 
they could do in their areas to help improve student 
performance. 

If the initial response was tepid from some 
quarters — McCormick heard resistance from people 
who said the tests had nothing to do with their 
responsibilities — it nonetheless provided a way to “get 
various people talking about the results and about how 
they’re involved and implicated in what’s going on 
here,” McCormick says, and to better understand that 
the mission of the college is “really all of our concern.” 

The CLA helped reinforce the college’s 
commitment to “intentionally develop students over 
the course of four years,” McCormick says. Those at 
Cabrini believe that their work with the CLA helped 
them articulate what they meant by providing a 
“developmental approach” to education. Long a part of 
the college’s rhetoric, that principle had been difficult 
to define until the CLA presented a snapshot of a 
value-added educational experience. Applying lessons 
learned from that assessment, the college was able to 
shape more deliberately a developmental curriculum 
for its general education program, keying in on the 
transferable skills — such as writing, critical thinking, 
problem solving, and analytical reasoning — that are 
measured by the CLA. 

“We’re still in the middle of playing out a 
developmental approach, but the CLA was, again, one 
of those anchors that helped us think about a general 
education program in just that way.” Moreover, he 
says, Cabrini faculty members who have worked with 
the CLA have started to envision how it could inform 
“a lot of opportunity for pedagogical change as well as 
curricular change.” 
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“The CLA sparked a reaffirmation of Cabrini’s mission.... Directly 
as a result of its work with the CLA, the college has been able to 
renew campus-wide discussions about how it transforms good 
students into great ones.” 

-Charlie McCormick, Dean for Academic Affairs, Cabrini College 



In some ways, too, the CLA sparked a reaffirmation 
of Cabrini s mission. Cabrini has a tradition of 
accepting students of modest abilities as well as more 
distinguished students. Directly as a result of its work 
with the CLA, McCormick says, the college has 
been able to renew campus-wide discussions about 
how it transforms good students into great ones. 
Accordingly, McCormick says, while Cabrini does not 
set out to recruit students based exclusively on prior 
academic achievement, it does intend to redouble its 
commitment to “create a structure for our students to 
achieve excellence.” 

“One of the most important things the CLA 
confirmed was the value of this institution,” 
McCormick says. “We live in an incredibly 
competitive market here in Philadelphia. Our next 
door neighbors are Eastern University, Villanova, 



Bryn Mawr, and Haverford. Swarthmore is just down 
the road. And it s easy to forget that we are doing 
something important. The CLA reminded us of that in 
a way that was very, very powerful.” 

Cabrini s work with the CLA now coincides with 
a confluence of factors that is helping the institution 
develop a culture of evidence and assessment. The new 
general education program calls for intentional and 
integrated assessment of student learning institution- 
wide. The college recently completed a strategic plan 
that calls for enhancing assessment initiatives in order 
to create a contemporary, innovative teaching and 
learning environment. The plan also includes the 
appointment of a coordinator of assessment. 

Cabrini s experience is but one more example of 
how institutions are working with — and learning 
from — the Collegiate Learning Assessment. 
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THE CHALLENGES OF USING THE CLA 



Learning how best to use CLA results is a work in 
progress, and the institutions that are pioneering 
it through the CIC/CLA Consortium have had to 
wrestle with a number of challenges — some minor, 
some significant. Application of the CLA results to 
improve teaching and learning continues to be refined. 

Virtually all the institutions in the Consortium 
have had to address faculty resistance to the CLA and 
have struggled to get students to take the CLA. On 
campuses where the CLA has been introduced through 
administrative channels (such as the president or 
vice president for academic affairs), faculty members 
resisted it because they perceived it as a “top- 
down” initiative. On other campuses, some faculty 




members have initially found it too time consuming, 
a distraction from other work, or have resisted efforts 
they perceive as moving toward “teaching to the test.” 
As with many campus discussions, greater success 
seems to come when there is shared commitment 
and transparency about efforts to assess and improve 
student learning. 

“I think that once you get faculty members to 
sit down and look at what the CLA is testing, they 
agree generally that this is a valuable test,” says 
Linda DeMeritt of Allegheny College. “It’s not that 
faculty members are leery of assessment,” she says, but 
rather that they are wary of anything that smacks of 
“teaching to the test.” Allegheny’s experience, she 
says, has been that “when they actually see a test like 
this, which is not your standard multiple choice test, 
they begin to see its value.” 

Another challenge has been faced by institutions 
with large populations of nontraditional students. The 
CLA uses SAT or ACT scores as benchmarks — tests 
that nontraditional students, particularly adult 
learners, may have taken decades ago or not at all. 

As noted earlier, Alaska Pacific University faced the 
challenge of having many seniors without either SAT 
or ACT scores and worked with CLA administrators 
to adopt the Scholastic Level Exam (SLE) as an 
alternative benchmark. Other Consortium members, 
including Centenary College in New Jersey, Heritage 
University in Washington state, and Indiana Wesleyan 
University, have faced similar challenges. 

While the CLA provides a reliable indication 
of an institution’s overall value-added contribution 
to student learning, it does not diagnose the 
factors that lead to results that are above, or below, 
expected performance. Put differently, the CLA is an 
outcome measure, but many of the factors affecting 
student learning are educational processes that the 
CLA does not measure and was not intended to 
capture. A promising solution to this limitation is to 
compare CLA scores for groups of students within an 
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institution. In order to get valid results with sub- 
groups, an institution must conduct in-depth sampling, 
perhaps doubling the number of students tested. 

With these larger samples, institutions can reliably 
do the investigative work that will determine what is 
contributing to group differences, develop appropriate 
interventions, and subsequently test the efficacy of 
those interventions. 

The logistics of getting students to sit at a computer 
long enough to take the test can be a dilemma when it 
comes to first-year students and an outright challenge 



when it comes to seniors. The timing of the test has 
also been something of an issue. President Richard 
Cook of Allegheny College describes the problem 
this way: “We tend to have tested the first-year 
students when they’re very fresh, excited, focused, 
undistracted,” while seniors take the test “when they’re 
dragging at the finish line.” Allegheny is studying the 
issue of when the test is administered. Cook says, to 
determine what timing would work best for obtaining 
the most meaningful comparison. 




CM Performance Tasks present students with a realistic scenario, providing a set of documents that can be used 
to prepare a response, such as memos, newspaper articles, tables of numbers, figures, research study abstracts, 
and reports. Students must determine which documents are more reliable and relevant and consider how the 
provided data might be best combined to answer the questions posed. 
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Getting Students to Take the CLA 

Figuring out the best means of getting students to 
take the CLA has also been challenging. Through 
a process of trial and error in the first years of the 
Consortium, institutions have settled on strategies that 
work well (see “Campus Strategies,” page 25). Each 
campus’ approach is different. Some use incentives, 
such as gift certificates to the book store, extra 
commencement tickets, a chance to win an iPod, or 
even cash — typically $25 or $35. Others have found 
that administering the CLA during orientation is a 
good way to get first-'year student participation. 

Indiana Wesleyan University experimented with a 
package of incentives that lets students who take the 
CLA register ahead of other students, get first crack in 
student drawings for housing, and even gain access to 
a premium parking lot. At one college, the assessment 
coordinator has been known to babysit for the children 
of seniors while they take the CLA. 

Westminster College in Missouri gives the CLA 
to all first-'year students during their first week on 
campus in the fall, then captures seniors as part of 
an annual spring Assessment Day, a Tuesday when 
regular classes are suspended to allow time for testing. 
The expectation there is that seniors will take the 
CLA that day. In 2007, 90 percent complied. William 
Woods University, coincidentally in the same town as 
Westminster, also has Assessment Day, albeit with a 
very large stick: students who fail to take the CLA that 
day are not permitted to continue in their majors. 

In its first year of using the CLA, Bethel University 
in Minnesota struggled during the fall semester to find 
a time when first-'year students could take the test. 
Administrators contacted some instructors and invited 
students to participate, and the university offered $5 
gift certificates as incentives, but in the end Bethel 
suffered from small samples, according to Richard 
Sherry, the university’s dean of faculty growth and 
assessment. In Bethel’s second and third years with 
the CLA, the university decided to make the test part 
of assessment activities during its first-'year welcome 
week. Out of pools of students whose ACT/SAT 
scores were representative of the entire class, students 



were assigned at random to take the CLA, while the 
remaining students were given a critical thinking 
assessment that Bethel has used for some years. “That 
worked extremely well, and we got our sample size 
in roughly two and a half hours,” Sherry reports. To 
get seniors into the test. Bethel University follows 
a strategy that many members of the Consortium 
employ — working directly with faculty and students in 
senior capstone classes, in this case courses that have 
intensive critical thinking and writing components. 

Faculty members at Bethel help motivate students 
to take the CLA in a variety of ways. Sherry says. 
Some simply tell students that taking the CLA is a 
course requirement; others make the CLA optional. 
Some instructors offer a small number of bonus points 
or the equivalent as a carrot for taking the test. 
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Campus Strategies 

Responses from a survey ofCIC/CLA Consortium members, February 2008, 
based on experiences covering the past three years 
(Due to multiple responses and rounding, items may not total 100 percent) 

Most successful approaches for recruiting first-year students: 

■ Embedded CLA administration in a first-year seminar or required first-year course (50%) 

■ Given during new student orientation (29%) 

■ Targeted appeals through faculty members or other advisors (14%) 

■ Financial incentives (7%) 

Least successful approach attempted to recruit first-year students: 

■ Open solicitations and appeals for volunteers 

Most successful approaches for recruiting seniors: 

■ Embedded in a senior seminar or capstone course (43%) 

■ Direct appeals to seniors (with or without monetary or gift incentives) (29%) 

■ Targeted appeals through faculty members (14%) 

■ During on-campus assessment day activities (7%) 

■ Embedded in other upper-level classes (7%) 

Least successful approaches attempted to recruit seniors: 

■ Open solicitations and appeals for volunteers (44%) 

■ Open solicitations and appeals for volunteers with monetary or gift incentives (33%) 

■ Monetary or gift incentives alone (22%) 

Incentives offered that have been most effective with first-year students: 

■ Retail gift card or gift certificate (29%) 

■ Cash (7%) 

■ Class extra credit (7%) 

■ Did not use incentives (50%) 

Incentives offered that have been most effective with seniors: 

(Among those that offered incentives, 45 percent did so through a raffle or drawing.) 

■ Retail gift card or gift certificate (36%) 

■ Extra graduation tickets and/or rebates of graduation fees (21%) 

■ Cash (14%) 

■ Class extra credit (7%) 

■ Did not use incentives (21%) 

Additional approaches used to recruit first-year and senior students: 

■ Personal letters from the president or dean (30%) 

■ Appeals from faculty members (30%) 

■ Classroom presentations by assessment committee members (26%) 
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At Texas Lutheran University, students take the 
CL A as part of the Freshman Experience Course, 
which all first-year students take in the fall semester. 
TLU administrators ask faculty members in a 
representative sample of sections of that course to 
allow testing of their students 
in one of their class periods. 

Cabrini College tried 
a lottery to get seniors to 
take the CL A; winners 
received letters inviting 
them to take the test. Many 
students ignored the letters, 
forcing the schools director 
of institutional research 
to follow-up individually. 

Cabrini elects not to use cash 
or prize incentives. 

When it first joined the 
Consortium, Lynchburg College got CLA off to a 
strong start when the dean of the college personally 
visited classrooms to talk with students and faculty 
members about the importance of the test. Ten-dollar 
gift cards were offered to students as inducements. 



Outside the classroom, the dean also met with faculty 
members, often over lunch, to discuss the value of 
the CLA. Some of Lynchburg’s faculty members have 
supported the CLA by giving students extra service 
points for taking the test. 

Clever faculty members at 
Seton Hill University built 
the CLA into the curriculum 
of senior capstone seminars. 
In one seminar, students take 
the CLA as part of studying 
why personal assessment is 
critical to lifelong learning. 
In a different seminar, 
students take the test as part 
of class discussions on the 
benefits of the liberal arts. 

One college had little 
success when it offered raffle 
tickets with an iPod as a prize but had better success 
when it offered a $100 rebate toward senior fees. Still, 
for many Consortium members, motivating seniors 
to take the test — and take it seriously — remains a 
challenge. 



Through a process of trial 
and error in the first years of 
the Consortium, institutions 
have settled on strategies 
that work well. Each campus’ 
approach is different. 




Consortium participants at 
the summer 2007 meeting 
confer on strategies for 
getting students to take 
the Coiiegiate Learning 
Assessment. 
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THE CIC/CLA CONSORTIUM: FUTURE DIRECTIONS 



I n 2007, with substantial new funding from The 
Teagle Foundation, CIC issued a call for proposals 
to extend and expand the Consortium s work for 
another three years, through the spring of 201 L Of the 
65 CIC member colleges and universities that applied 
to participate in this next phase of the Consortium, 47 
were selected, including 27 currently participating. 

The work of the CIC/CLA Consortium in the next 
phase will be in response to two key lessons from the 
project thus far, says CIC senior vice president and 
project director, Harold V. Hartley IIL “First, we have 
found that engaging faculty members in using the CLA 
is absolutely essential to improving student learning. 
Second, pairing CLA results with other assessment 
measures, such as NSSE or portfolio analysis, provides 
more robust diagnostic information to use in targeting 
areas for improving teaching and student learning.” 
Both of these lessons, taken to heart, will help foster a 
culture of evidence on campus, says Hartley. 

In this new phase, members of the CIC/CLA 
Consortium will use the CLA to push the assessment 
of valued added in promising new directions. The 
emphasis will be on developing more comprehensive 
campus assessment plans for improving teaching and 
learning by incorporating additional measures such 
as student portfolios, campus-based instruments, 
and other standardized tests, like NSSE, to provide 
multiple sources of evidence in addition to the CLA. 
Experience has shown that while the CLA provides a 
reliable measure of overall institutional contribution 
to student learning, it is most beneficial when used 
in conjunction with other efforts. Accordingly, 
institutions in the Consortium are being encouraged to 
develop comprehensive assessment programs tailored 
to campus needs and competencies. 

Some institutions will also conduct in-depth 
sampling, testing additional students that match 
characteristics of interest to the institution — division, 
major, gender, or race/ethnicity, for example — to allow 
a more nuanced analysis. 



Consortium members will continue to explore the 
potential synergy between the CLA and NSSE. Unlike 
the CLA, which is an outcomes assessment, NSSE 
examines the process components of student learning. 
Pairing the two instruments can provide important 
clues to areas for improving student learning. 

In the new stage of the project, continuing 
Consortium members will mentor institutions using 
the CLA for the first time. The original members of 
the Consortium have now developed a significant body 
of knowledge about the CLA. Their experience will be 
invaluable in helping colleges and universities new to 
the CLA adopt the assessment on their own campuses. 

Finally, the new stage of the Consortium will see 
member institutions working to integrate faculty 
members more fully into the assessment of student 
learning. The focus will be on making use of CLA 
results to improve pedagogy, redesign curriculum, and 
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create a campus culture favorable to assessment efforts. 
A new CAE program, dubbed CLA in the Classroom, 
will enable faculty members to use a mock-version of 
the CLA in their classes as a means of gauging and 
improving student skills. The program guides faculty 
members through the process of developing their own 
performance tasks modeled after the CLA. 

In many ways, CLA in the Classroom represents 
a next logical phase in the development of the CLA. 
Developed with significant input from members of 
the CIC/CLA Consortium, CLA in the Classroom 
gives faculty members a set 
of CLA-related tools — such 
as scoring rubrics and test 
questions patterned after 
the CLA prompts — that 
apply some of the principles 
of CLA directly to help 
improve student learning. 

“1 was able to preview an 
early version of the CLA in 
the Classroom program at 
the 2007 annual meeting 
of the Consortium,” said 
Marc Chun, senior research 
scientist at CAE and director 
of CLA in the Classroom, 

“and this was incredibly 
helpful for getting early feedback that helped in 
refining elements of this initiative.” 

An important development, and essentially a 
giant step forward in building practical ways to 
enrich outcomes-based student learning, CLA in the 
Classroom will equip individual faculty members with 
CLA-based tasks, prompts, test questions, and rubrics 
through which they can collect their own diagnostic 
information about student strengths and weaknesses. 
Laculty members will use that information to help 
students understand why they achieved the scores 
they did and what to do next to improve their critical 
thinking, analytic reasoning, problem solving, and 
written communication skills. The information will 
provide a means, says Chun, for faculty members to 



“have a better conversation with their students about 
where their performance could be improved relative 
to higher-order skills.” The new instructional tools 
will provide faculty members with the practical means 
to do diagnostic work with students — and become 
more personally invested in an assessment-based 
commitment to improve student learning. 

Because faculty members haven’t necessarily been 
explicitly trained to teach with these higher-order 
skills in mind, the success of CLA in the Classroom 
is predicated to a large extent on a significant culture 

shift. In essence, the program 
equips faculty members who 
may be content experts but 
may not have strong training 
in pedagogy to take greater 
responsibility for assessing 
what students are learning 
and to help them improve. 

Among the institutions in 
the CIC/CLA Consortium, 
CLA in the Classroom sparks 
significant excitement. 
Cabrini College pilot-tested 
this instrument in July 
2007 and in March 2008 
hosted the first CLA in 
the Classroom Academy to 
train faculty and staff from across the country. When 
Charlie McCormick, dean for academic affairs, and 
his colleagues reported on their experience at the 
Consortium’s summer 2007 meeting, the enthusiasm 
about it was palpable. Lisa Ratmanksy, who directs 
Cabrini’s center for teaching and learning, summarized 
the promise of this evolving approach. The questions 
that CLA raises, she said, “are allowing faculty 
members to become much more intentional about the 
pedagogical underpinnings of their work.” 

Lor example, Cabrini College is looking to use CLA 
in the Classroom to bolster course-based assessment. 
The college is also looking to engage faculty members 
in programmatic assessment of its new general 
education program. Paving the way, Cabrini first plans 



CLA in the Classroom equips 
faculty members who may 
be content experts but may 
not have strong training in 
pedagogy to take greater 
responsibility for assessing 
what students are learning 
and to help them improve. 
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to help faculty understand the history, philosophy, 
and methodology behind the CLA and its scoring 
rubrics. A planned 2008 summer academy will engage 
faculty members in CLA concepts and constructs. 
Additionally, Cabrini will continue to participate 
in the formal administration of the CLA at the 
institutional level. 

Other members of the Consortium are building 
on their overall experience to advance their use of 
the CLA as well. Like Cabrini, Allegheny College 
plans summer faculty development programs based 
on CLA concepts — specifically the performance task 
and analytic writing task models — to link the CLA 
directly with academic disciplines and interdisciplinary 
courses. Allegheny will also continue to cross-'analyze 
multiple measures of student outcomes to learn more 
about its students’ performance. 

Barton College, meanwhile, intends to continue 
to build on what it is learning from its work with the 
CLA to assist the college in strategic planning and 
in improving its academic program. In particular, the 
college wants to focus increased attention on student 
skills in writing and critical thinking and to construct 
curricular elements that give students more practice in 
these skills. 

The CLA has sparked Alaska Pacific University 
to engage its entire faculty in a new effort to define 
desired student competencies for each of its academic 
programs and to develop rubrics for measuring student 
achievement against the desired competencies. 

Continuing to move aggressively on the path it 
has set for itself, the University of Charleston plans 
in the coming year to compare longitudinal data from 



the CLA, NSSE, and MAPP (Measure of Academic 
Proficiency and Progress) instruments and then assess 
portfolios of student work in the context of these 
standardized instruments. The university also plans 
to use results from the CLA to inform the review of 
academic program and assessment processes. 

Among other institutional plans, Westminster 
College (MO) will continue its efforts to cross ^ analyze 
CLA data with results from NSSE. Building in large 
part on its experience with the CLA, Lynchburg 
College is moving to develop an academic program 
that will put more focus squarely on desired outcomes 
of student knowledge. Texas Lutheran University is in 
the process of launching a quality enhancement plan 
that identifies specific learning outcomes for its general 
education program, solidifies its program review 
procedures, develops an institution-' wide assessment 
plan, and refines ways to make all of these processes 
work together — activities broadly informed by insights 
from the institution’s experiences with the CLA. 

These institutional strategies show that the 
members of the CIC/CLA Consortium are now poised 
to draw on lessons derived from CLA data and to apply 
principles of assessment in practice. The institutions 
will review CLA scores, develop hypotheses about the 
factors influencing student learning, and then seek 
ways to test these hypotheses. They will look to merge 
multiple sources of evidence (such as NSSE, licensure 
and entrance exam scores, and portfolio assessments) 
with the expert judgment of faculty members to 
determine ways to improve student learning. They 
will continue to collaborate with one another to 
share discoveries and effective strategies for the 
improvement of student learning. 



“When you create a new testing paradigm like the CLA, there is no 
better way to understand its benefits than by working with faculty 
who are in the classroom.” 

-Roger Benjamin, President, Councii for Aid to Education 
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Breakout sessions during the 
CIC/CLA Consortium meeting 
in August 2007 focused on 
institutionai practices and 
strategies for using the CLA 
to improve teaching and 
iearning. 



In short, members of the CIC/CLA Consortium 
will now move to refine their approaches to assessing 
student learning outcomes and to engage faculty 
members more deeply in this work. “When you create 
a new testing paradigm like the CLA,” said Roger 
Benjamin, president of CAE, “there is no better way 
to understand its benefits than by working with faculty 
who are in the classroom. We thank CIC for giving us 
this opportunity.” 

“The lessons we have learned in working closely 
with the CIC/CLA Consortium — about issues such as 
student recruitment, sampling design, combining data, 
and reporting — have been incorporated into the work 
that improves the program for all institutions,” says 
Chun. 

A fundamental truth inherent in the expansive and 
complex entity of higher education is that colleges and 
universities will always be able to find ways to improve 
their work. Speaking at the summer 2005 meeting of 
the CIC/CLA Consortium, Richard Hersh suggested 
that “assessment is a way of teaching and learning.” 
Hersh said that the CLA “empowers colleges to be 
more efficacious and successful in getting students 
to learn what we value” and is therefore “a powerful 



institutional change tool.” In that same spirit, Hersh 
told participants in the 2007 summer meeting of the 
Consortium that “we have to be better diagnosticians 
when it comes to teaching and learning” and that in 
that context, “assessment becomes a powerful way of 
teaching.” 

As the experiences of the members of the CIC/ 
CLA Consortium show, the lessons that assessment 
has to teach are varied and rich. While on one level 
the commitment of CIC/CLA Consortium members 
is to the hard task of applying and learning from a 
workable, meaningful measure of student learning, the 
test itself is just one part of a broader effect. In many 
ways Consortium members are also working to create 
campus cultures based on evidence of learning. And 
in many distinctive ways — as we have seen in this 
report — the work to build an evidence-based culture is 
having a broadly transformative effect on the whole of 
the institution. 

By using the Collegiate Learning Assessment to 
assess student outcomes and improve teaching and 
learning, the institutions in the CIC/CLA Consortium 
are at the vanguard of important educational reform. 
Their work embodies true academic leadership. 
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MEMBERS OF THE CIC/CLA CONSORTIUM 



Consortium Members 2004-2005 

Allegheny College (PA) 

Bethel University (MN) 

Chatham College (PA) 

College of Saint Benedict/ 

Saint Johns University (MN) 

Consortium Members 2005-2008 

Alaska Pacific University 
Allegheny College (PA) 

Aurora University (IL) 

Averett University (VA) 

Barton College (NC) 

Bethel University (MN) 

Cabrini College (PA) 

Centenary College (NJ) 
Charleston Southern 
University (SC) 

College of Saint Benedict/ 

Saint Johns University (MN) 
Franklin Pierce University (NH) 

Consortium Members 2008-2011 

Alaska Pacific University 
Allegheny College (PA) 

Aurora University (IL) 

Averett University (VA) 

Barton College (NC) 

Bethel University (MN) 

Cabrini College (PA) 

Carlow University (PA) 
Charleston Southern 
University (SC) 

College of Notre Dame (MD) 
College of Saint Benedict/ 

Saint Johns University (MN) 
Dominican University (IL) 

Drake University (lA) 

Franklin Pierce University (NH) 
Hilbert College (NY) 



Hampshire College (MA) 
Hendrix College (AR) 
Mary wood University (PA) 
Millikin University (IL) 
Pace University (NY) 



Heritage University (WA) 
Indiana Wesleyan University 
Loyola University New 
Orleans (LA) 

Lynchburg College (VA) 
Marian College (Wl) 

Pace University (NY) 

Pacific University (OR) 

Seton Hill University (PA) 
Southwestern University (TX) 
Stonehill College (MA) 

Texas Lutheran University 
University of Charleston (WV) 



Illinois College 
Jamestown College (ND) 

John Carroll University (OH) 
Juniata College (PA) 

LaGrange College (GA) 

Loyola University New 
Orleans (LA) 

Lynchburg College (VA) 
Marian College (Wl) 
Morningside College (lA) 
Nebraska Wesleyan University 
Pace University (NY) 

Seattle Pacific University (WA) 
Seton Hill University (PA) 
Southwestern University (TX) 
Springfield College (MA) 
Stephens College (MO) 



Seattle Pacific University (WA) 
University of Charleston (WV) 
Westminster College (UT) 



University of Evansville (IN) 
University of Great Falls (MT) 
Ursinus College (PA) 

Ursuline College (OH) 

Wagner College (NY) 

Wartburg College (lA) 

Wesley College (DE) 
Westminster College (MO) 
Westminster College (UT) 
William Woods University (MO) 



Stonehill College (MA) 

Texas Lutheran University 
The College of St. 

Scholastica (MN) 

The University of Findlay (OH) 
Trinity Christian College (IL) 
University of Charleston (WV) 
University of Evansville (IN) 
University of Great Falls (MT) 
Upper Iowa University 
Ursinus College (PA) 

Ursuline College (OH) 

Wagner College (NY) 
Westminster College (MO) 
Westminster College (UT) 
Willamette University (OR) 
William Woods University (MO) 



THE COUNCIL OF INDEPENDENT COLLEGES *31 




RESOURCES AND RESEARCH 



Resources 

Council of Independent Colleges-For additional information about the Council of Independent Colleges, visit 
ClC’s website at wwwxic.edu. 

CIC/CLA Consortium-For additional information about the CIC/CLA Consortium, visit CIC s website at www. 
cic . edulprojects _ser vices! coops! cla . asp. 

Council for Aid to Education-Maintains a rich collection of articles about the CLA and related topics at the 
following link: www.cae . org! content! pro _collegiate_reports_publications . htm. 

Collegiate Learning Assessment-For additional information about the Collegiate Learning Assessment, visit the 
CAE website at www.cae. or g!content!pro_collegiate. htm. 

Teagle Foundation-The Teagle Foundation s commitment to promoting and strengthening liberal education 
grounds all of its grantmaking. For more information, visit www.teaglefoundation.org. 



Research and Articles about the CLA 

For more information about the CLA, including its conceptual and theoretical foundations, these select articles 
and papers may be of interest: 



“The CLA Contribution to the Improvement of Teaching and 
Learning in Higher Education,” by Roger Benjamin, February 14, 
2008. (Available online at www.cae.org/content/pro_collegiate_ 
reports_publications . htm) 

“The Collegiate Learning Assessment: A Tool for Measuring the 
Value Added of a Liberal Arts Education,” by Anne Grosso de 
Leon, Carnegie Results, Fall 2007. 

“No Gr_du_te Left Behind,” by James Traub, The New York 
Times, September 30, 2007. 

“Going Naked,” by Richard Hersb, Peer Review, Spring 2007. 

“The Collegiate Learning Assessment: Facts and Fantasies,” 
by Stephen Klein, Richard Sbavelson, Roger Benjamin, and 
Roger Bolus, 2007. Evaluation Review, Vol. 31, No. 5, 415-439. 
(Available online at www.cae.org/content/pro_collegiate_reports_ 
publications.htm) 

“The Group Approach,” by Karen W. Arenson, The New York 
Times, November 13, 2006. 



“Beyond the Rankings: Measuring Learning in Higher 
Education,” by Gene I. Maeroff, The Hecbinger Institute, 2006. 
(Available online at www.teaglefoundation.org/learning/pdf/2006_ 
hechinger.pdf) 

“Principles for the Uses of Assessment in Policy and Practice: 
President’s Report to the Board of Trustees of the Carnegie 
Foundation for the Advancement of Teaching,” by Lee S. 
Sbulman, 2006. (Available online at www. teagle foundation, 
org/learning/pdf/ 2006_shulman_assessment.pdf) 

“What Does College Teach?” by Richard Hersb, Atlantic Monthly, 
November 2005. 

“Liberal Arts Colleges: Taking the Lead on Assessment and 
Accountability,” by Richard Hersb, Liber alArtsOnline, January 
2005. (Available online at http://liberalarts.wabash.edu/cila/home. 
cfm?news_id=2 171) 

“Responding Responsibly to the Frenzy to Assess Learning In 
Higher Education,” by Richard Sbavelson and Leta Huang, 
Change, January/February 2003. 
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